Scaling data streams with Catmandu and linked data fragments

C

File-based storage of Digital Objects and constituent datastreams: XMLtapes and Internet Archive ARC files

This paper introduces the write-once/read-many XMLtape/ARC storage approach for Digital Objects and their constituent datastreams. The approach combines two interconnected file-based storage mechanisms that are made accessible in a protocol-based manner. First, XML-based representations of multiple Digital Objects are concatenated into a single file named an XMLtape. An XMLtape is a valid XML file; its format definition is independent of the choice of the XML-based complex object format by which Digital Objects are represented. The creation of indexes for both the identifier and the creation datetime of the XML-based representation of the Digital Objects facilitates OAI-PMH-based access to Digital Objects stored in an XMLtape. Second, ARC files, as introduced by the Internet Archive, are used to contain the constituent datastreams of the Digital Objects in a concatenated manner. An index for the identifier of the datastream facilitates OpenURL-based access to an ARC file. The interconnection between XMLtapes and ARC files is provided by conveying the identifiers of ARC files associated with an XMLtape as administrative information in the XMLtape, and by including OpenURL references to constituent datastreams of a Digital Object in the XML-based representation of that Digital Object.Comment: 12 pages, 1 figures (camera-ready copy for ECDL 2005

The multi-faceted use of the OAI-PMH in the LANL Repository

This paper focuses on the multifaceted use of the OAI-PMH in a repository architecture designed to store digital assets at the Research Library of the Los Alamos National Laboratory (LANL), and to make the stored assets available in a uniform way to various downstream applications. In the architecture, the MPEG-21 Digital Item Declaration Language is used as the XML-based format to represent complex digital objects. Upon ingestion, these objects are stored in a multitude of autonomous OAI-PMH repositories. An OAI-PMH compliant Repository Index keeps track of the creation and location of all those repositories, whereas an Identifier Resolver keeps track of the location of individual objects. An OAI-PMH Federator is introduced as a single-point-of-access to downstream harvesters. It hides the complexity of the environment to those harvesters, and allows them to obtain transformations of stored objects. While the proposed architecture is described in the context of the LANL library, the paper will also touch on its more general applicability

DRIVER Technology Watch Report

This report is part of the Discovery Workpackage (WP4) and is the third report out of four deliverables. The objective of this report is to give an overview of the latest technical developments in the world of digital repositories, digital libraries and beyond, in order to serve as theoretical and practical input for the technical DRIVER developments, especially those focused on enhanced publications. This report consists of two main parts, one part focuses on interoperability standards for enhanced publications, the other part consists of three subchapters, which give a landscape picture of current and surfacing technologies and communities crucial to DRIVER. These three subchapters contain the GRID, CRIS and LTP communities and technologies. Every chapter contains a theoretical explanation, followed by case studies and the outcomes and opportunities for DRIVER in this field

The aDORe Federation Architecture

The need to federate repositories emerges in two distinctive scenarios. In one scenario, scalability-related problems in the operation of a repository reach a point beyond which continued service requires parallelization and hence federation of the repository infrastructure. In the other scenario, multiple distributed repositories manage collections of interest to certain communities or applications, and federation is an approach to present a unified perspective across these repositories. The high-level, 3-Tier aDORe federation architecture can be used as a guideline to federate repositories in both cases. This paper describes the architecture, consisting of core interfaces for federated repositories in Tier-1, two shared infrastructure components in Tier-2, and a single-point of access to the federation in Tier-3. The paper also illustrates two large-scale deployments of the aDORe federation architecture: the aDORe Archive repository (over 100,000,000 digital objects) at the Los Alamos National Laboratory and the Ghent University Image Repository federation (multiple terabytes of image files).Comment: 43 pages, 4 figures, 2 table

OpenURL syntax description: version OpenURL/1.0f, 2000-05-16

In order to enable the delivery of context-sensitive services or -- initially bibliographic -- metadata, information providers are invited to add an OpenURL to the metadata, when it is being displayed as a result of a search/browse in their information systems. The OpenURL is designed to enable the transfer of the metadata from the information service to a service component that can provide context-sensitive services for the transferred metadata